COVID-19 stigmatization after the development of effective vaccines: Vaccination behavior, attitudes, and news sources

Objective To compare COVID-19 stigmatization at two pandemic time points (1) August 2020—during lockdowns and prior to vaccine rollout, and (2) May 2021—during vaccine rollout, when approximately half of U.S. adults were vaccinated. Methods Comparison of COVID19-related stigmatization and associated factors in two national internet surveys conducted in August 2020 (N = 517) and May 2021 (N = 812). Factors associated with endorsing stigmatization were identified using regression analysis. The main outcomes included endorsement of stigmatization and behavioral restrictions towards persons with COVID-19 and towards persons of Chinese descent. A previously developed “stigmatizing attitudes and behavioral restrictions” scale was adapted to measure the intersection of negative attitudes toward COVID-19 disease and negative attitudes toward persons of Chinese descent. Results COVID-19 related stigmatization declined significantly from August 2020 to May 2021. Many factors were associated with stigmatizing in both surveys: full time employment, Black race, Hispanic ethnicity, worry about contracting COVID-19, probable depression, and Fox News and social media as sources of information (all positively associated), and self-assessed knowledge about COVID-19, contact with Chinese individuals, and publicly funded news as sources (all negatively associated). Positive attitudes toward vaccination were associated with stigmatization. Conclusions COVID-19 related stigmatization reduced substantially over these two points in the pandemic, with many continuities in the factors associated with stigmatizing. Despite the reduction in stigmatizing, however, some stigmatizing attitudes for both COVID-19 and Chinese individuals remained.

contains Personal Health Information (PHI) which is protected under the Health Insurance Portability and Accountability Act (HIPPA), such as vaccination status and whether the individual suffers from any of the "underlying conditions" that would be likely to make a COVID infection more severe, as well as demographic characteristics. Providing public access to the data used in our analyses could threaten loss of the confidentiality of PHI, and was not to be permitted according to our proposal submitted to the IRB. Access to the data can be provided through an approved Data Use Agreement between our institution (New York University) and the institution with which the user is affiliated. Persons wanting to access the data should communicate with the NYU IRB (email contact: irbinfo@nyulangone.org) to initiate a Data Use Agreement. amount of misinformation and disinformation about COVID-19 that has been disseminated in the U.S., often through social media platforms [20][21][22][23].
In this study, we used non-random national internet samples to compare the COVID-19 stigmatization U.S. at two different points in the epidemic. The first survey was conducted in August 2020, when various lockdowns had been implemented [24]. The second survey occurred in May 2021, a time when approximately half of U.S. adults had received at least one dose of a COVID-19 vaccine, and the Centers for Disease Control and Prevention (CDC) had just announced guidance that fully vaccinated persons no longer needed to wear masks in most settings [25]. This time period also coincided with increasing public polarization over the vaccines [26,27]. We consider two primary questions in the comparison of the two surveys: 1) Did the overall frequency of stigmatization change? and 2) Did the factors associated with expressing stigmatization remain the same or change? Identifying changes and consistencies in stigmatization within a rapidly changing pandemic may provide insights into the nature of disease-related stigmatization and possibly for preparing for future epidemics and pandemics. The Health Belief Model (HBM) is used to interpret the findings [28,29]. This model is frequently used in health communications, in which the perceived threat (seriousness and susceptibility) of the disease is emphasized as a rationale for engaging in protective behaviors. The "protective behaviors" may include avoiding contact with and placing behavioral restrictions on persons with the disease. However, given the controversies around the vaccines and the mis-and dis-information about the vaccines, we did not attempt to use the HBM to predict how stigmatization and how the correlates of stigmatization might have changed from pre-to post development of the vaccines.

Data collection
Amazon's Mechanical Turk (MTurk) is a crowdsourcing platform that provides access to an on-demand and diverse global workforce, an estimated 225,000 of which are U.S.-based [30]. Through MTurk, researchers and other "requesters" can post tasks that workers can complete if they meet pre-specified criteria. Participants in this study were eligible to complete the online survey if they were at least 18 years of age, had previously completed at least 500 MTurk tasks, and had approval ratings greater than or equal to 90% for previously completed MTurk tasks. The first survey was launched on August 5, 2020, during a period of intense lockdowns and social distancing. The second survey was launched on May 21, 2021, soon after the CDC announced new guidance that fully vaccinated persons no longer needed to wear masks or practice social distancing outdoors or in most indoor settings, excluding health care facilities, while flying or taking public transit, and in congregate settings [25]. Persons who had not been vaccinated, however, were still advised to wear masks in almost all situations outside of private homes. The May 2021 survey included all the items in the August 2020 survey, and additional items about having been vaccinated and intentions to become vaccinated.
Data quality was assessed using two attention checks requiring participants to select a specific response option and a time-to-completion analysis. Given the significant outliers, we implemented a minimum acceptable time to complete the survey of one standard deviation below the whole sample mean (151.562 seconds for the August 2020 survey, 182.897 seconds for May 2021 survey), and a maximum cutoff score of 1,000 seconds for the August 2020 survey and 1,400 seconds for the May 2021 survey. Of the 1,606 respondents between the two surveys, a final sample of 1,312 (August 2020 N = 517; May 2021 N = 812) was retained for this analysis. Entries were excluded due to failed attention checks (N = 51), unacceptable completion times (N = 64), and having incomplete data (N = 179).

Study variables
As a primary purpose of the 2021 survey was to compare COVID-19 related stigmatization at two points in the pandemic-during initial lockdown vs. after vaccine rollout-we used many of the items from the 2020 survey in the 2021 survey. Table 1 presents the major COVID-19 related items in our two surveys.

Primary outcome
The primary outcome of interest was COVID-19 related stigmatization, measured with the Stigmatizing Attitudes and Behavioral Restrictions (SABR) Scale. This SABR scale was adapted from a 2003 study comparing stigmatization of SARS-1 and HIV/AIDS [31]. Despite the many differences in the epidemiology of SARS-1 and HIV/AIDS (in modes of transmission, in case fatality rates, geographic areas of concentration, social groups with which the diseases were associated, total numbers of cases) there were strong similarities in the patterns of responses to the SABR items for SARS-1 and HIV/AIDS.
Responses to all of the individual SABR items were significantly correlated across the two diseases, and many of the same factors were associated with stigmatizing both SARS-1 and HIV/AIDS. These associated factors included education, income, race/ethnicity, greater worry about contracting the disease, less knowledge of the disease, and mental health problems [31].
Exploratory factor analysis of the August 2020 survey showed a strong single general factor explaining approximately 71% of the variance, with all items loading 0.6 or greater onto this factor. Reliability for this scale was excellent (Cronbach's alpha = 0.90). This SABR scale should be considered as measuring the intersection of stigmatization of COVID-19 and stigmatization of persons of Chinese descent. It was not intended to capture all aspects of stigmatization of either the disease or of the social group.

Contact with persons with or associated with COVID-19 and with Chinese persons
These items were adapted from stigma research [32] suggesting that personal contact would decrease stigmatization.

Experiences of discrimination related to COVID-19 and race/ethnicity
We included questions on having experienced stigmatization to examine whether experiencing stigmatization might lead to empathy for persons associated with COVID-19 and less stigmatizing or might lead to resentment and increased stigmatizing.

Other COVID-19 related questions
Several additional questions were adapted from Health Belief Model [28,29] theory (which posits that a perceived threat of a disease will lead to greater motivation to actions that would reduce the threat of the disease) and the 2003 study that suggested the threat value of the disease (worry about contracting the disease) will be associated with efforts to avoid contact (placing behavioral restrictions on persons with or associated with the disease). Probable depression was also considered likely to increase the threat value of the disease. Knowledge about the disease is clearly an important aspect of beliefs about the disease, and our 2020 survey found that self-assessed knowledge was associated with reduced stigmatization. New items were added on having underlying conditions that would increase the likelihood of severe COVID-19 illness and contact with persons who had severe COVID-19 as we hypothesized these would increase the perceived severity of the COVID-19.  (3,4) Have you ever known somebody who became seriously ill or died from COVID-19 Have you ever known somebody who identifies as Chinese?

Experience of Discrimination related to COVID-19
Have you experienced stigmatization or discrimination related to COVID-19?  (3) Have you experienced stigmatization or discrimination because of your race/ethnicity

Vaccination Behavior and Intentions
Have you been or are you in the process of being vaccinated against COVID-19, and if not, do you intend to receive a COVID-19 vaccination in the future 1. Vaccinated 2. Definitely will get vaccinated 3. Probably will get vaccinated 4. Probably will not get vaccinated 5. Definitely will not get vaccinated 1. Vaccinated (1) 2. Definitely/probably will get vaccinated (2,3) 3. Probably will not get vaccinated (4) 4. Definitely will not get vaccinated (5) Further combined as 1.Pro-vaccines (1, 2, 3) and 2. Anti-vaccines (4, 5)

Other COVID-19 related questionnaire items
Do you have any of the underlying conditions, such as diabetes, being overweight, heart disease, lung/breathing diseases, that could make COVID-19 disease more severe The vaccination behavior and intentions items were added to the May 2021 survey to examine the relationship of being vaccinated/not being vaccinated to stigmatizing.

Preferred news source
These items were used in the 2020 survey and were strongly associated with stigmatization in that survey. That preferred news sources are associated with beliefs about COVID-19 has been noted in many previous studies [36][37][38].

Lack of hypotheses
We did not formulate hypotheses for our two major research questions (whether the prevalence of stigmatization or the correlates of stigmatization would change after development of vaccines) because we believed that the amount of polarization, mis-and dis-information about the vaccines precluded straightforward application of any theoretical framework.

Statistical analysis
Cross tabulations and chi square tests assessed bivariate relationships between the August 2020 and May 2021 surveys and demographic characteristics, vulnerabilities to and experiences with COVID-19, and the SABR scale, with p-values of less than 0.05 used to detect statistically significant differences. We calculated crude odds ratios to examine associations of possible predictor variables with the COVID-19 SABR items using univariate logistic regression. All predictors and covariates were moved into multivariable logistic regression models to estimate adjusted odds ratios. The vaccination status/vaccination intentions variable formed a continuum from positive to very negative attitudes toward vaccination, and the Cochran-Armitage test for trend was used to assess relationships with COVID-19 related attitudes and experiences and the SABR items.
As noted below, there were modest differences in the demographic characteristics of the respondents in the two surveys. We thus used demographic characteristics as control variables in all multivariable analyses. Analyses were performed in STATA version 17 [39]. The study was approved by the New York University Institutional Review board. All participants provided written informed consent after reading a summary of the study, but before beginning the survey. Table 2 presents the demographic characteristics of respondents in the two surveys. There were modest differences in the demographic characteristics of the participants in the two surveys. Participants in the May 2021 survey were moderately older, less likely to be non-Hispanic Black or Hispanic, and less likely to have college degrees. Table 3 presents endorsements of the five SABR items in the two surveys. In the May 2021 survey, there was less endorsement of the SABR items, including for the scale as a whole and for each of the individual items. All the differences in the SABRs were in the direction of less stigmatization in the May 2021 survey.

Results
The changes in stigmatization occurred within the context of changes in many other aspects of the COVID-19 pandemic. Table 4 presents information on experiences with and potential vulnerabilities to COVID-19 among the respondents. There were multiple differences in the responses to these survey items. Participants in the May 2021 survey were less likely to be worried about contracting COVID-19, more likely to assess themselves as knowledgeable, more likely to report contact with persons who had COVID-19, less likely to report probable depression, and less likely to report social media as a source of information. We then conducted a bivariable logistic regression of the odds of stigmatizing in the May 2021 survey compared to the odds of stigmatizing in the August 2020 survey. The OR was 0.45 (95% CI 0.36-0.56). We also conducted a multivariable logistic regression to control for the demographic and "vulnerabilities and experiences" variables in our survey; the adjusted OR was 0.52 (95% CI 0.37-0.72) showing that the reduction in endorsing SABR items remained significant after controlling for the other variables in our survey. The odds for endorsing at least one SABR item were 48% lower in the May 2021 survey compared to the August 2020 survey. (Full data available from the first author).

Factors associated with endorsing SABRs in August 2020 and May 2021 surveys
We used logistic regression to identify factors associated with the SABR scale in the two surveys. Table 5 presents the bivariable and multivariable models for each of the two surveys. Overall, there were many variables that were significant in at least one of the regressions and substantial similarities in the factors that were significantly associated with the SABR scale across the two surveys. The variables significantly associated with stigmatization in both surveys were: full time employment, Black race, Hispanic ethnicity, worry about contracting  received at least one inoculation by the time of the May 2021 data collection. We therefore used our "vaccination status/vaccination intentions" variable for further analyses of the May 2021 survey data. As shown in Table 5, pro-vaccine attitudes (vaccinated, definitely will get vaccinated and probably will get vaccinated) versus anti-vaccines (probably will not get vaccinated and definitely will not get vaccinate) were associated with endorsing stigmatization. Table 6 presents the SABR items by the vaccination status/intentions variable. The SABR scale as a whole and all the individual SABR items showed significant differences across vaccination status/intentions. The direction of the differences was consistent in that being vaccinated and having positive intentions to be vaccinated were associated with a greater likelihood of endorsing an individual SABR item. The "definitely will not get vaccinated" group was distinctive in their low likelihood of endorsing any of the SABR items. Table 7 presents the relationships of the vaccination status/intention variable with selected "COVID-19 experiences and vulnerabilities" items that were associated with endorsing/not endorsing the SABR scale. There were significant relationships between the vaccination status/ intentions variable and COVID-19 worry, less knowledge, having underlying conditions, and contact with severe COVID-19 cases and probable depression.

Discussion
As noted in the Introduction, the primary purposes for the comparison of the pre-versus postvaccine development surveys were to assess 1) whether the frequency of endorsing COVID-19 related stigmatization changed, and 2) whether the factors associated with endorsing COVID-19 related stigmatization changed.

Reduction in stigmatization
There was a substantial decrease in stigmatization, both in the crude odds ratio and in the multivariable adjusted odds ratio. This can be seen as a cause for some optimism with respect to decreasing COVID-19 related stigmatization in the US.

Continuity in factors associated with endorsing stigmatizing attitudes
The second major finding from the comparison of the 2020 and 2021 surveys were the multiple factors associated with stigmatization in both surveys. These included full time employment, Black race, Hispanic ethnicity, worry about contracting COVID-19, probable depression, and Fox News and social media as preferred sources of information (all positively associated), and self-assessed knowledge about COVID-19, contact with Chinese individuals, and publicly funded news as preferred sources (all negatively associated).

The Health Belief Model and the SABR scale
The Health Belief Model (HBM) posits that greater perceived threat of a disease-specifically greater perceived severity and greater perceived susceptibility-will lead to greater motivation to actions that would reduce the threat of the disease [29]. Endorsing behavioral restrictions on persons with or associated with a threatening disease would thus be a straightforward prediction from the model. A number of the factors associated with stigmatization in Table 4 are consistent with the HBM. For example, worry about contracting COVID-19 and having underlying conditions that would increase the likelihood of severe disease were positively associated with stigmatizing. Greater perceived knowledge (less uncertainty) about COVID-19 and more experience with COVID-19 were negatively associated with stigmatizing. The changes between the two surveys were also generally consistent with the HBM. For example, worry decreased between the surveys and stigmatization also decreased.

Vaccination behaviors and intentions
The biggest change across the two surveys was the emergence of vaccination behavior and new attitudes towards vaccination, At the time of the second survey, being vaccinated or intending to be vaccinated was positively associated with endorsing stigmatizing attitudes and behavioral restrictions. At first, this may appear counterintuitive. Public health officials might have hoped that being vaccinated would greatly allay worries about contracting COVID-19. There are, multiple possible reasons for the finding.
First, our data should not be interpreted as showing an effect that vaccination did not reduce stigmatization. We did not have data from the same individuals prior to and after vaccination so that we cannot infer causation at the individual level.
The prevalence of stigmatization among all respondents in the 2020 survey was 66.0%, and the prevalence of stigmatization among respondents in the 2021 survey was 46.4%. (See Table 3). Thus, there was a net reduction of approximately 20% in survey participants endorsing stigmatizing attitudes and behavioral restrictions between the 2020 and 2021 surveys. In the 2021 survey 557/812 (69%) of the participants were vaccinated. It is possible that the 20% reduction in the prevalence of stigmatizing all came from the 31% of 2021 participants who were not vaccinated, but it would seem more likely that there was also a net reduction in endorsing stigmatization among the 2021 participants who were vaccinated.
Second, there are several plausible reasons for why those had been vaccinated might still have been quite concerned about contracting COVID-19 at the time of the second survey. The vaccines were never presented as completely effective. At the time the second survey, the CDC was still recommending that vaccinated persons wear masks in many public settings [25]. Also, the vaccinated persons did have higher prevalence of underlying conditions, more personal experience with severe cases of COVID-19, and less self-perceived knowledge (greater uncertainty) about COVID-19, all of which could contribute to the threat value of COVID-19 and lead to endorsing stigma and behavioral restrictions even if one was vaccinated.
Third, as part of the increasing polarization over the vaccines, there were increasing news stories raising doubts about the effectiveness of the vaccines and concerns about the potential harmful side effects associated with being vaccinated, which could have increased the perceived threat of COVID-19 among those who had been vaccinated [40].
Each of these factors would be consistent with the basic premise of the Health Belief Model that greater perceived threat of COVID-19 would be likely to motivate endorsing stigmatizing attitudes and behavioral restrictions on persons who associated with COVID-19.

News sources
There has been varying coverage of COVID-19 among different news sources in the U.S., with some sources taking an approach based on science while others have taken a more xenophobic and anti-scientific approach. Specific news sources, particularly Fox News, significantly downplayed the COVID-19 pandemic, particularly in the early stages (late 2020-early 2021) and were skeptical of much of the scientific expertise surrounding the virus and transmission of COVID-19 [41,42].
However, Fox News also spread many xenophobic messages about COVID-19. Fox News frequently referred to COVID-19 as the "Chinese Coronavirus" which acted to reinforce the negative stereotypes of Asian Americans. For examples, Fox News carried stories that the Chinese government "intentionally" released the COVID-19 virus [43][44][45] These xenophobic stories likely contributed to some of the associations seen in the analysis related to Chinese individuals (questions 3-5 in the SABR scale items). While Fox News did downplay the importance of COVID-19 as a disease, which would be associated with lower stigmatization of persons associated with the disease, Fox News also emphasized the association of COVID-19 with China, which would have increased stigmatization of Chinese persons. In our sample, the xenophobic, anti-Chinese effect appears to have been much stronger that the downplay of COVID-19 effect.

Potential causal pathways
Many of the odds ratios in the univariable logistic regressions were attenuated in the multivariable regression models, suggesting complex associations, with many variables having both direct and indirect associations with endorsing stigmatizing attitudes and behavioral restrictions. Having longitudinal individual-level data would be useful in developing a full conceptual model of causal pathways (with specification of confounders, mediators, and moderators) for endorsing stigmatizing attitudes and behavioral restrictions. Ideally, individual-level data would be matched with significant developments in the pandemic: initial concern and lockdowns, development of vaccines, political polarization of vaccination and mask mandates, emergence of multiple variants, and emergence of "pandemic fatigue."

The SABR scale
The SABR scale was originally developed for studying stigmatization of SARS-1 and AIDS. These were serious diseases; the case fatality rate for SARS-1 was about 14% [46] and until the development of the "cocktail" antiretroviral treatments for AIDS, its case fatality rate was over 90% [47].
HIV infection was (and still is) lifelong. At the time we first used the scale for COVID-19, August 2020, the case fatality rate was unknown, but the numbers of COVID-19 deaths in the US were quite high, approximately 450-500 per day [48].
We are currently updating the SABR scale to increase its applicability to less serious infectious diseases, such as COVID-19, which now has a very low case fatality rate.
The data in this report provide evidence for multiple aspects of construct validity for the scale. Higher worry about contracting COVID-19 was associated with greater likelihood of endorsing stigmatizing attitudes and behavioral restrictions on persons with or associated with COVID-19 which may be considered construct validity for attitudes. The scale also had construct validity for experiences, with previous exposure to severe cases of COVID-19 and having underlying conditions associated with greater endorsement of stigmatizing attitudes and behavioral restrictions. The scale also had construct validity for behavior-as deciding that one would definitely not get vaccinated can be considered behavior-and was associated with very low SABR scale scores.
The scale also showed construct validity with stigmatization theory [14] as greater contact with persons of Chinese descent as associated with less endorsement of stigmatization and behavioral restrictions of persons of Chinese descent.
This short scale cannot be considered as a comprehensive measure of the stigmatization of an infectious disease associated with a socially devalued group, but there is substantial evidence for its construct validity.

Future research
While many of the relationships in our data are generally consistent with the Health Belief Model, we would not propose that this model can generate a full understanding of the causal pathways leading to stigmatization of COVID-19. As noted in the introduction, COVID-19 stigmatization was added to existing anti-Asian/anti-Chinese stigmatization in the U.S. and was followed by a large increase in anti-Asian hate crimes [9]. We conceived of our scale as measuring the intersection of anti-Chinese stigmatization and anti-COVID-19 stigmatization and not to measure separate components. Future research should involve adapting the scale to changes in knowledge about COVID-19, for example it clearly has a much lower case-fatality rate than SARS-1 or AIDS, but there is also the potential for symptoms persisting over long periods of time (long COVID-19) [49].
The scale also needs to be complemented with measures of factual knowledge of COVID-19, the sources of accurate, mis-and disinformation and of group affiliations. And as noted above, individual level longitudinal data, tied to developments in the epidemiology, psychology, and politics of the pandemic, are needed.

Limitations
Several limitations of this study should be mentioned. First, we did not have a nationally representative sample in either survey. We did, however, control for demographic characteristics in our multivariable analyses. Second, we did not have the same respondents in both surveys, so we were unable to study changes in attitudes and stigmatization at the individual level. Third, we had only a single item measuring knowledge of COVID-19 and this was based on selfassessment. We were not able to assess the factual content of the respondents' self-assessed knowledge. Having an accurate assessment of the respondents' knowledge of COVID-19 and of the vaccines might be quite important in assessing relationships between news sources and stigmatizing.
These limitations are important, but we believe that our data do show an important reduction in endorsing stigmatizing attitudes and behavioral restrictions, the general applicability of the Health Belief Model, and the importance of different media as preferred news sources. The development and politicization of the vaccines has led to an additional layer of complexity in the stigmatization of persons who have COVID-19 and groups who are associated with COVID-19.

Conclusions
There was a very substantial reduction in the endorsement of stigmatizing attitudes and behavioral restrictions between the August 2020 and May 2021 surveys. Many of the factors associated with stigmatizing were significant in both surveys. The development and partial implementation of effective vaccines did not eliminate stigmatizing and added new complexity to the patterns of stigmatization.
Generating enough public concern about a new threat to health to lead people to take appropriate actions while minimizing stigmatization of persons having or associated with the disease is always a difficult public health communication task. Achieving this balance may be especially difficult in a context of widespread mis-and disinformation, a variety of competing news sources, political polarization, and the potential for hate crimes.